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ABSTRACT 

Model-critical  procedures  provided  a  means  to 
scrutinize  an  assumed  parametric  statistical  model  by 
varying  the  way  the  data  Is  processed  for  repeated 
fits  to  the  model.  The  criticism  of  the  data  Is 
iccompllshed  using  the  generalized  likelihood 
function  for  the  assumed  probability  density  of  the 
lata.  The  degree  of  criticism  Is  controlled  by  a 
user  specified  constant  c.  The  model-critical 
parameter  estimates  are  obtained  by  maximization  of 
the  generalized  likelihood  function.  When  e  »  0,  no 
rrltlclsm  Is  performed  and  maximum  likelihood 
•stlmates  are  obtained.  These  procedures  can 
indicated  If  any  model  assumptions  have  been 
violated.  Model  critical  estimation  procedures  are 
aresented  for  autoregressive  (AR)  models..  The 
inalysls  of  an  AR  example  Is  presented,  /cl.vvvrrn*-  '_J 

INTRODUCTION  .C^'P  > 

Signal  models  are  as  the  name  Implies  only  models 
of  reality.  As  such,  they  only  approximate  the  true  I 
process  and  In  general,  more  than  one  model  can  be 
used  to  describe  a  given  set  of  data.  Under  these 
conditions,  the  experimenter  must  determine  models 
which  adequately  describe  the  observed  data. 

Depending  on  how  the  model  Is  to  be  used,  one  of  the 
candidate  models  Is  chosen.  The  selected  model  Is 
then  analyzed  as  to  how  well  It  describes  the 
observed  data.  In  this  paper,  the  term  model  refers 
to  the  structural  and  distributional  description  of 
the  data.  The  nominal  distributional  model  Is  the 
Gaussian  distribution;  the  structural  models  are 
parametric  models  such  as  an  autoregression.  This 
paper  examines  model-critical  procedures  which 
scrutinize  the  data  and  the  assumed  model  by  varying 
the  way  the  data  Is  processed  during  model  fitting. 

Some  aspects  of  model-critical  analysis  have  been 
presented  In  [11,  12,  13.  14.  15,  16]. 

Model-critical  analysis  Is  based  on  the 
generalized  likelihood  [12]  for  a  random  sample  y-j , 

V2.  •••  yn 


n 

Lc(e)  -  (i/c)  ^jfc(yi:e)/04(e)  -ij 


(l) 


where 

a  -  c/(l+c),  f ( y-t ;e)  Is  the  assumed  probability 
density  of  the  data  evaluated  at  observation 
yi,  «  Is  a  parameter  vector  that  characterizes 
the  data  and  the  density  f. 
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controlled  by  c.  Using  L'HospItals  rule,  the  limit  of 
Lr(6)  as  c  approaches  zero  Is  L0(9)  the  usual  log 
likelihood  for  9.  Differentiating  Lc(9)  with 
respect  to  e  and  setting  the  result  equal  to  zero 
yields 


»le(9) 


fC(yV  e)  [8l°_9  ,f(y1;9)  1  aloo  0(91 

q*(e)  L  *  <1+c>  a® 

which  Is  a  necessary  condition  that  must  be  satisfied 
by  9(c).  For  the  models  discussed  here,  equation  (3) 
Is  used  to  obtain  9(c).  From  (3)  It  can  be  seen  that 
each  term  In  the  sum  Is  weighted  by  the  c-th  power  of 
the  assumed  probability  density  of  the  data  evaluated 
at  y<(.  The  value  of  c  determines  the  amount  and 
type  of  weighting  used  In  (3).  Positive  values  of  c 
downweight  outlying  observations  and  negative  values 
of  c  downweight  Inlying  observations.  This  weighting 
produces  a  criticism  of  the  data  and  the  assumed  model 
f(y;»)  that  can  Indicate  If  any  model  assumptions  have 
been  violated. 

Outliers  are  an  example  of  a  violation  of  the 
model  assumptions  on  the  data.  It  Is  noted  that  an 
observation  is  an  outlier  only  with  respect  to  the 
assumed  underlying  model;  If  a  different  model  Is 
used,  the  observation  may  no  longer  be  an  outlier. 
Unlike  unstructured  data,  where  an  outlier  ‘sticks 
out*,  the  structure  of  a  model  can  hide  the  outlier. 

If  multiple  outliers  are  present,  they  can  compensate 
each  other  [2],  in  time  series  where  the  observations 
are  not  Independent,  outliers  need  only  be  large  with 
respect  to  the  error  process  to  seriously  affect  the 
parameter  estimates  [8].  With  outliers  of  this 
magnitude,  they  may  not  show  up  In  plots  of  the  data. 
Fox  [4]  considers  two  outlier  models  for  time  series. 
The  additive  outlier  Is  a  gross  error  at  a  single 
observation.  The  Innovations  outlier  Is  a  large  value 
in  the  error  process  due  to  a  heavy  tailed  error 
distribution.  It  Is  noted  that  both  types  of  outliers 
can  occur  with  Independent  as  well  as  dependent 
observations.  With  dependent  observations,  the 
innovative  outlier  will  affect  subsequent  observations 
due  to  the  correlation  between  observations.  A  host 
.>f  robust  or  resistant  procedures  are  available  to 
reduce  the  effect  of  outliers  on  the  parameter 
estimates  [1,10].  The  previous  discussion  has  focused 
on  outlying  contamination;  however.  Inlying  or 
short-tailed  contamination  can  also  be  a  concern  [6]. 


Q(9) 
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f1+c(y;9)  dy 


(2) 


Is  the  Information  generating  function  for  f(y;9) 

[5,  12],  and  c  Is  the  model-critical  parameter.  The 
model-critical  estimate  for  e  Is  the  value  of  9(c) 
which  maximizes  (1).  The  estimate  9(c)  Is  a  robust 
estimate  for  9  with  the  degree  of  robustness 


From  the  above  discussion.  It  can  be  seen  that 
robustness  and  goodness  of  fit  are  related. 
Model-critical  analysis  uses  this  relationship  to 
examine  models  for  a  given  set  of  data.  The  analysis 
compares  maximum  likelihood  and  robust  parameter 
estimates.  Clearly,  the  robust  and  maximum  likelihood 
estimates  must  estimate  the  same  quantity  If  they  are 
to  be  comparable.  From  the  derivation  of  Lc(9),  the 
estimate  9(c)  Is  a  consistent  estimate  of  9  [3], 

Thus,  9(c)  and  9(o)  are  two  consistent  estimates  of  9. 
If  the  data  and  the  assumed  model  are  Internally 


consistent,  then  e( o)  and  e(c)  should  be  approximately 
equal  over  a  range  of  c  values.  However,  If  the  data 
and  the  assumed  model  are  not  consistent,  then  e(c) 
will  change  considerably  as  c  Increases.  Large 
changes  In  parameter  estimates  e(c)  Indicate  that  the 
model  requires  closer  examination.  As  an  H-estlmator 
[7],  robust  weights  are  obtained  as  part  of  the 
estimation  process.  For  c  w  0,  these  critical  weights 
can  be  used  to  flag  questionable  observations. 
Examination  of  the  weights  aids  the  analyst 
In  evaluating  the  model.  For  example,  small  weights 
Indicate  outlying  contamination  when  c  >  0. 

MODEL-CRITICAL  EST1HAT10H 

Let  the  density  f(y;«)  In  (1)  be  the  r-varlate 
Gaussian  density 


f(y)  -  |2*D|  "1/2  exp[-  (y-m)T  D^Cy-m)^)  (4) 

with  mean  vector  m  and  covariance  matrix  D.  For  the 
Gaussian  density  In  (4),  It  Is  straight  forward  to 
evaluate  (2}  and  obtain 


Q(m.D.c)  -  t|2*D|  C  (l*c)rf1/2.  c  >  -  1.  (5) 

Using  (3),  (4)  and  (5),  the  following  set  of  Implicit 
estimation  equations  for  m  and  0  are  obtained. 


The  errors  c|  are  assumed  to  be  Independent  and 
Identically  distributed  6auss1an  random  variables  with 
zero  mean  and  covariance  matrix  0.  For  the  model 
given  by  (9),  the  generalized  likelihood  without  the 
constant  term  denoted  L(c)  Is 


n 

L( c )  -  (1/c)  ^  Aexp  (-  c  eiD“1e1/2)  (10) 

1-1 


where 


A  -  |(l+c)/2*0|  0.5c/(l+c)> 

t\  -  yi  -  htx^e). 


For  the  proposed  model  h(x;e)  In  L(c),  the 
model-critical  estimates  of  ©  and  0  are  obtained  by 
maximizing  (10)  over  e  and  D.  For  many  models, 
setting  equal  to  zero  the  derivatives  of  L(c)  with 
respect  to  ©  and  0  yields  a  set  of  Implicit  equations 
which  can  be  solved  via  a  fixed  point  algorithm. 

ANALYSIS  OF  AUTOREGRESSIVE  MODELS 

For  the  multivariate  autoregressive  (AR)  model  of 
order  p,  (9)  can  be  expressed  as 


m  - 


n 


23wiyi 
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(6)  «r  yi 


HD 


n 

0  "  (1+c)  ^3w1(y1  "  m)(yl  "  ra,T 
1-1 


where 


for  1  •  p+1,  p+2,  ....  n.  The  model -critical 
estimates  for  0  and  A^,  k-1,  2,  ...,  p  are  obtained 
by  differentiating  L(c)  with  respect  to  0  and  A^, 

k-1,  2 . p;  setting  the  derivatives  equal  to  zero; 

and  solving  for  D  and  A^,  k-1,  2,  ....  p.  The 
model-critical  estimation  equations  are 


fc<yk)/SfC(yt) 


1-1 


(8) 


A(c)  -  G_1b 


and 


(12) 


The  estimates  for  m  and  0  are  the  values  m(c)  and  D(c) 
which  satisfy  (6)  and  (7).  Clearly,  when  e  -  0,  m(0) 
and  0(0)  are  the  usual  maximum  likelihood  estimates. 
The  specification  of  the  user-provided  constant  c  Is 
based  on  sample  size  n,  dimension  r,  and  the  character 
of  the  sample. 

For  models  with  structure  In  addition  to  a  mean 
vector  and  covariance  matrix,  the  errors  can  be 
expressed  as 


n 

0(c)  -  (0*c)/w.)  ^2  wi  *i<c>  ei(c>T  <13> 
1-pel 

where 

A(c)  -  [ A.j (c) ,  A2(c) . Ap(c) )T  .  and  (14) 


«1  -  yi  -  h(*i;®) 

where 


(9) 


e^e)  -  (y, 


Vc)  y1-k)- 


(15) 


*1  Is  a  r  x  l  vector  of  errors, 

yi  Is  a  r  x  1  vector  of  observations, 

xi  Is  a  p  x  1  vector  of  concomitant  variables, 

and 

©  Is  a  p  x  1  vector  of  parameters. 


The  block  matrix  entries  or  b  are 


9U  "  £  Vk-1yI-j* 


k-pel 


1.  j  -  1.2 . P 


.  ,  .  .  .  .  .  For  c  -  0.4.  Figure  4  Is  a  plot  of  the 

and  the  block  matrix  entries  of  b  are  model-critical  weights 


n 

bi  ■  2  wl-i  •  1  •  . p 

k-p+1 


where 


W|(  -  exp(-  c  ek(c)T  0(c)'1  ek(c)/2)  (16) 


and 

n 

w-  ■  2  wk  • 
k-p+1 

The  estimates  A(0),  0(0)  and  A(c),  0(c),  c  w  0  are 
conditional  maximum  likelihood  and  conditional 
model -critical  estimates  since  they  are  conditioned  on 
the  first  p  observations;  however,  the  estimates  will 
still  be  referred  to  as  maximum  likelihood  or 
model-critical  estimates. 

As  an  Illustration,  model-critical  estimates  are 
presented  for  a  simulated  univariate,  Gaussian  AR(4) 
process  with  representation  given  by  (11)  where 
A]  -  2.0625,  A2  -  -2.4325,  A?  -  1.5845, 

A4  »  -0.652.  and  the  xt  are  Independent 
Identically  distributed  with  zero  mean  and  unit 
variance.  Figure  1  Is  a  plot  of  the  realization  used 
to  the  obtain  parameter  estimates.  Table  1  contains 
the  parameter  estimates  for  c  -  0,  0.1,  0.2,  0.3,  and 
0.4.  It  can  be  seen  that  the  model-critical  and 
maximum  likelihood  estimates  are  approximately 
equal.  Next,  additive  outliers  were  added  to  four 
observations  selected  at  random  In  the  realization 
shown  In  Figure  1.  The  outliers  are  Independent, 
identically  distributed  Gaussian  random  variables 
with  zero  mean  and  variance  2;  also,  the  outliers  are 
Independent  of  y*.  The  additive  outlier  model  (9] 

Is  given  by  z*  -  y*  +  vt  where  y*  Is  the 
AR( 4)  process  and  v*  Is  the  outlier.  For  this 
example,  v*  w  0  for  only  four  observations.  Figure 
2  Is  a  plot  of  the  data  In  Figure  1  with  outliers 
added  to  four  observations;  without  a  priori 
knowledge,  one  would  not  suspect  that  outliers  are 
present.  Martin  [9]  notes  that  for  time  series,  the 
outliers  need  only  be  large  relative  to  the 
Innovations  process  **  to  seriously  affect  the 
parameter  estimates.  With  outliers  of  this  magnitude, 
they  may  not  stand  out  as  they  do  when  the 
observations  are  Independent.  This  Is  clearly  seen  In 
Table  2,  which  presents  the  maximum  likelihood  and 
model-critical  parameter  estimates  for  the  data  In 
Figure  2.  As  c  Increases,  the  model-critical 
estimates  approach  the  true  values;  the  Improvement  In 
the  estimates  follows  from  the  downweighting  of  the 
outliers  In  model-critical  estimation.  For  the  data 
with  outliers.  Figure  3  contains  plots  of  the 
model-critical  and  maximum  likelihood  spectrum.  The 
critical  spectrum  Is  closer  to  the  true  spectrum  (the 
solid  line)  than  the  maximum  likelihood  spectrum 
estimate.  For  the  data  with  outliers,  the  maximum 
likelihood  spectrum  contains  more  high  frequency 
components  than  the  critical  spectrum.  This  Is  the 
case  because  maximum  likelihood  estimation  fits  all 
the  data,  whereas  critical  estimation  fits  the  bulk  of 
the  data  without  outliers. 


wt  -  exp(-  c(zt  -  ^  ak(c)zt_k)2/2s2(c)) 
k-1 

where  s2(e)  and  a^c)  are  calculated  using  (12) 
and  (13).  Since  the  critical  weights  are  a  measure  of 
fit  between  the  data  and  the  model,  analysis  of  the 
weights  Is  an  Integral  part  of  the  modeling  process. 

In  Figure  4,  the  small  weights  at  observations  5,  6, 
11,  and  25  Indicate  that  the  model  which  describes  the 
bulk  of  the  data  does  not  give  a  good  fit  to  these 
observations.  In  fact,  some  of  the  subsequent  weights 
are  small  due  to  the  dependence  between  observations. 
Small  weights  alert  the  experimenter  to  the  fact  that 
further  analysis  of  the  data  and  model  may  be 
necessary. 

CONCLUSION 

Model-critical  procedures  have  been  presented  for 
analyzing  the  joint  character  of  the  data  and  the 
assumed  model.  The  critical  estimation  equations  for 
multivariate  autoregressive  models  were  presented. 

The  model-critical  weights  which  are  obtained  during 
parameter  estimation  can  be  used  to  Identify 
observations  which  violate  the  model  assumptions.  It 
is  noted  that  model-critical  procedures  can  be  applied 
to  wide  variety  of  models.  These  procedures  can 
assist  the  engineer  In  model  Identification,  and 
analysis  of  the  error  process. 
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Maximum  Likelihood  and  Model-Critical  Parameter 
Estimates  for  a  Simulated  Univariate  AR(4) 
Process  with  Sample  Size  •  100 


c 

«1 

«2 

43 

*4 

S? 

0.0 

2.1346 

-2.5685 

1.7113 

-0.7092 

0.9100 

0.1 

2.1190 

-2.5438 

1.6906 

-0.7094 

0.9137 

0.2 

2.0984 

-2.5040 

1 .6527 

-0.6981 

0.8915 

0.3 

2.0841 

-2.4820 

1.6353 

-0.6999 

0.8663 

0.4 

2.0631 

-2.4400 

1.5962 

-0.6914 

0.8241 

TABLE  2 

Maximum  Likelihood  and  Model-Critical  Parameter 
Estimates  for  a  Simulated  Univariate  AR(4) 
Process  with  Four  Additive  Outliers,  and 
Sample  Size  ■  100 


c 

•1 

»2 

•3 

44 

s2 

0.0 

1.7661 

-1.7643 

0.9633 

-0.4409 

2.0344 

0.1 

1.8477 

-1.9116 

1 .0799 

-0.4651 

1 .7752 

0.2 

1 .9556 

-2.1520 

1.2968 

-0.5410 

1.4180 

0.3 

2.0342 

-2.3064 

1.4354 

-0.5912 

1.1620 

0.4 

2.0391 

-2.3498 

1 .4765 

-0.6102 

1.0740 
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Figure  V.  A  Simulated  AR(4)  Process  with  Innovations 
Distributed  Normal(O.l). 
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